Skip to content

Add cell-aware cross-compilation arguments#239

Merged
dpc merged 1 commit into
rustshop:masterfrom
dpc:dpc/jj-swtvruqzuxmp
Jul 22, 2026
Merged

Add cell-aware cross-compilation arguments#239
dpc merged 1 commit into
rustshop:masterfrom
dpc:dpc/jj-swtvruqzuxmp

Conversation

@dpc

@dpc dpc commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Posted by Tau

Summary

Add a cell-aware argsFor resolver to craneMultiBuild. Projects can select the correct dependency artifacts for each existing target cell without replacing Flakebox's manual Cargo-cross model or changing its output matrix.

Details

argsFor runs once per toolchain/target cell before profile mapping. Its result flows through the existing overrideArgs/mergeArgs path.

Built-in cells expose explicit dependency-role package scopes:

  • nativeBuildInputs through pkgsBuildHost
  • buildInputs through pkgsHostTarget
  • depsBuildBuild through pkgsBuildBuild

Native cells expose the corresponding native roles. Android and wasm32-unknown-unknown do not pretend to have a Nixpkgs target-package universe; unsupported role access fails lazily with the cell and missing role. Existing callers and custom toolchains remain unchanged when they do not use scoped packages.

The change documents resolver semantics and merge precedence. Evaluation coverage includes native and AArch64 role selection, custom toolchains, unsupported-target diagnostics, capability filtering, Darwin evaluation, argument precedence, and once-per-cell evaluation.

The x86_64→AArch64 behavioral check uses a small OpenSSL Rust binary. It proves that argsFor selects pkgsBuildHost.pkg-config and pkgsHostTarget.openssl, executes the selected cross pkg-config wrapper, completes Crane dependency and package builds, and verifies:

  • the result is an AArch64 ELF;
  • it has a NEEDED libssl entry;
  • its RUNPATH selects target OpenSSL;
  • the selected libssl is also AArch64.

The earlier cross-linker blocker is fixed independently by merged PR #240; this change contains no linker-policy workaround.

Reviewers can validate the change with nix develop -c selfci check --candidate swtvruqz.

Comment on lines +6 to +11
mergeArgs =
left: right:
left
// right
// {
buildInputs = (left.buildInputs or [ ]) ++ (right.buildInputs or [ ]);

@dpc dpc Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file seems to be redoing bunch of stuff we already have. Is there point to it that I'm missing? If so, it should be documented and explained.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted by Tau

You’re right that the original fixture duplicated too much. Its non-obvious purpose was to run a nested evaluator: pure Nix cannot tell us how many times argsFor was forced, and builtins.tryEval cannot inspect exception text.

I replaced it with a smaller documented adapter that only supplies args, scalar overrideArgs, and mapWithProfiles. Package-role, capability, and merge behavior now stays on the existing real enhanced-Crane test path.

dpc added a commit to dpc/flakebox that referenced this pull request Jul 22, 2026
Add one optional, cell-aware `argsFor` resolver to `craneMultiBuild` so projects can select dependency artifacts for each existing target cell without changing Flakebox's manual Cargo-cross model. Resolve common project arguments once per cell before profile mapping and merge them through the existing `overrideArgs`/`mergeArgs` path. Preserve existing callbacks, selectors, output paths, and behavior when callers omit the resolver.

### Details

Attach explicit dependency-role contexts to built-in toolchains. Native cells expose the native `pkgsBuildHost`, `pkgsHostTarget`, and `pkgsBuildBuild` roles; ordinary Nixpkgs cross cells expose those roles from their cross package set. Android and `wasm32-unknown-unknown` expose no false target universe, and `capabilities.targetBuildInputs` is derived from the availability of the target-library role. Missing role access reports the toolchain/cell and role lazily, so custom toolchains without a context remain usable until their resolver requests scoped packages.

Document the public resolver contract, existing merge precedence, and the explicit-artifact boundary of the manual Cargo-cross model. Add evaluation coverage for native and x86_64-to-AArch64 role selection, custom/wasm/Android contextual diagnostics, capability-based filtering, three-layer argument precedence, Darwin portability, and exactly one resolver invocation per cell rather than per profile. The observable invocation-count and exception-text contracts use a compact, documented nested-evaluator adapter because pure Nix cannot expose either property; all package-role, capability, and merge behavior remains tested against the real enhanced Crane library.

A comprehensive architecture, judgment, maintainability, and reliability review initially found Darwin-incompatible test selection, inaccurate `depsBuildBuild` merge documentation, weak invocation/capability/diagnostic coverage, and duplicated capability state. Those issues were fixed and re-review passed. A later inline review correctly noted that the first evaluator fixture reproduced too much Crane and merge infrastructure; it was replaced with the minimal adapter described above. The maintainability, reliability, and judgment re-review passed with no remaining findings.

`nix build -L .#checks.x86_64-linux.craneMultiBuildTests`, all four supported-system focused check evaluations, `nix flake check --no-build --no-write-lock-file`, treefmt, and selfci pass. An attempted OpenSSL/pkg-config AArch64 build reached target OpenSSL discovery through the selected role scopes, then hit the existing linker policy at the time: AArch64 `ld.bfd` rejected `-Wl,--compress-debug-sections=zstd`. The behavioral link check remains outside this dependency-selection change rather than making PR rustshop#239 depend on unrelated linker/LLVM work.
@dpc
dpc force-pushed the dpc/jj-swtvruqzuxmp branch from 1ca628a to 6054298 Compare July 22, 2026 07:06
Add one optional, cell-aware `argsFor` resolver to `craneMultiBuild` so projects can select dependency artifacts for each existing target cell without changing Flakebox's manual Cargo-cross model. Resolve common project arguments once per cell before profile mapping and merge them through the existing `overrideArgs`/`mergeArgs` path. Preserve existing callbacks, selectors, output paths, and behavior when callers omit the resolver.

### Details

Attach explicit dependency-role contexts to built-in toolchains. Native cells expose the native `pkgsBuildHost`, `pkgsHostTarget`, and `pkgsBuildBuild` roles; ordinary Nixpkgs cross cells expose those roles from their cross package set. Android and `wasm32-unknown-unknown` expose no false target universe, and `capabilities.targetBuildInputs` is derived from the availability of the target-library role. Missing role access reports the toolchain/cell and role lazily, so custom toolchains without a context remain usable until their resolver requests scoped packages.

Document the public resolver contract, existing merge precedence, and the explicit-artifact boundary of the manual Cargo-cross model. Add evaluation coverage for native and x86_64-to-AArch64 role selection, custom/wasm/Android contextual diagnostics, capability-based filtering, three-layer argument precedence, Darwin portability, and exactly one resolver invocation per cell rather than per profile. The observable invocation-count and exception-text contracts use a compact, documented nested-evaluator adapter because pure Nix cannot expose either property; all package-role, capability, and merge behavior remains tested against the real enhanced Crane library.

Add an x86_64-to-AArch64 behavioral check with a small OpenSSL Rust binary. It asserts that `argsFor` selects `pkgsBuildHost.pkg-config` and `pkgsHostTarget.openssl`, executes the selected pkg-config wrapper, completes Crane dependency and package builds, and inspects the resulting binary, its `NEEDED` libssl entry, its target-OpenSSL `RUNPATH`, and libssl itself as AArch64 ELF artifacts.

A comprehensive architecture, judgment, maintainability, and reliability review initially found Darwin-incompatible test selection, inaccurate `depsBuildBuild` merge documentation, weak invocation/capability/diagnostic coverage, and duplicated capability state. Those issues were fixed and re-review passed. A later inline review correctly noted that the first evaluator fixture reproduced too much Crane and merge infrastructure; it was replaced with the minimal adapter described above, and re-review passed. Review of the behavioral check found that its first path assertion was not confined to `RUNPATH`; the check now extracts and verifies that entry specifically, and final re-review passed with no remaining findings.

`nix build -L .#checks.x86_64-linux.craneMultiBuildTests`, `nix build -L .#checks.x86_64-linux.cellAwareCrossInputs`, all four supported-system focused check evaluations, `nix flake check --no-build --no-write-lock-file`, treefmt, and selfci pass. The behavioral link check became feasible after the independent cross-linker compression fix merged on master; this change contains no linker-policy workaround.
@dpc
dpc force-pushed the dpc/jj-swtvruqzuxmp branch from 6054298 to 7d1834a Compare July 22, 2026 08:22
@dpc
dpc merged commit 35e15e4 into rustshop:master Jul 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant